173 research outputs found
Feedback Generation for Performance Problems in Introductory Programming Assignments
Providing feedback on programming assignments manually is a tedious, error
prone, and time-consuming task. In this paper, we motivate and address the
problem of generating feedback on performance aspects in introductory
programming assignments. We studied a large number of functionally correct
student solutions to introductory programming assignments and observed: (1)
There are different algorithmic strategies, with varying levels of efficiency,
for solving a given problem. These different strategies merit different
feedback. (2) The same algorithmic strategy can be implemented in countless
different ways, which are not relevant for reporting feedback on the student
program.
We propose a light-weight programming language extension that allows a
teacher to define an algorithmic strategy by specifying certain key values that
should occur during the execution of an implementation. We describe a dynamic
analysis based approach to test whether a student's program matches a teacher's
specification. Our experimental results illustrate the effectiveness of both
our specification language and our dynamic analysis. On one of our benchmarks
consisting of 2316 functionally correct implementations to 3 programming
problems, we identified 16 strategies that we were able to describe using our
specification language (in 95 minutes after inspecting 66, i.e., around 3%,
implementations). Our dynamic analysis correctly matched each implementation
with its corresponding specification, thereby automatically producing the
intended feedback.Comment: Tech report/extended version of FSE 2014 pape
IST Austria Technical Report
Board games, like Tic-Tac-Toe and CONNECT-4, play an important role not only in development of mathematical and logical skills, but also in emotional and social development. In this paper, we address the problem of generating targeted starting positions for such games. This can facilitate new approaches for bringing novice players to mastery, and also leads to discovery of interesting game variants.
Our approach generates starting states of varying hardness levels for player 1 in a two-player board game, given rules of the board game, the desired number of steps required for player 1 to win, and the expertise levels of the two players. Our approach leverages symbolic methods and iterative simulation to efficiently search the extremely large state space. We present experimental results that include discovery of states of varying hardness levels for several simple grid-based board games. Also, the presence of such states for standard game variants like Tic-Tac-Toe on board size 4x4 opens up new games to be played that have not been played for ages since the default start state is heavily biased
FlashProfile: A Framework for Synthesizing Data Profiles
We address the problem of learning a syntactic profile for a collection of
strings, i.e. a set of regex-like patterns that succinctly describe the
syntactic variations in the strings. Real-world datasets, typically curated
from multiple sources, often contain data in various syntactic formats. Thus,
any data processing task is preceded by the critical step of data format
identification. However, manual inspection of data to identify the different
formats is infeasible in standard big-data scenarios.
Prior techniques are restricted to a small set of pre-defined patterns (e.g.
digits, letters, words, etc.), and provide no control over granularity of
profiles. We define syntactic profiling as a problem of clustering strings
based on syntactic similarity, followed by identifying patterns that succinctly
describe each cluster. We present a technique for synthesizing such profiles
over a given language of patterns, that also allows for interactive refinement
by requesting a desired number of clusters.
Using a state-of-the-art inductive synthesis framework, PROSE, we have
implemented our technique as FlashProfile. Across tasks over large
real datasets, we observe a median profiling time of only s.
Furthermore, we show that access to syntactic profiles may allow for more
accurate synthesis of programs, i.e. using fewer examples, in
programming-by-example (PBE) workflows such as FlashFill.Comment: 28 pages, SPLASH (OOPSLA) 201
Automatic Generation of Alternative Starting Positions for Simple Traditional Board Games
Simple board games, like Tic-Tac-Toe and CONNECT-4, play an important role
not only in the development of mathematical and logical skills, but also in the
emotional and social development. In this paper, we address the problem of
generating targeted starting positions for such games. This can facilitate new
approaches for bringing novice players to mastery, and also leads to discovery
of interesting game variants. We present an approach that generates starting
states of varying hardness levels for player~ in a two-player board game,
given rules of the board game, the desired number of steps required for
player~ to win, and the expertise levels of the two players. Our approach
leverages symbolic methods and iterative simulation to efficiently search the
extremely large state space. We present experimental results that include
discovery of states of varying hardness levels for several simple grid-based
board games. The presence of such states for standard game variants like Tic-Tac-Toe opens up new games to be played that have never been
played as the default start state is heavily biased.Comment: A conference version of the paper will appear in AAAI 201
Synthesizing number transformations from input-output examples.
Abstract. Numbers are one of the most widely used data type in programming languages. Number transformations like formatting and rounding present a challenge even for experienced programmers as they find it difficult to remember different number format strings supported by different programming languages. These transformations present an even bigger challenge for end-users of spreadsheet systems like Microsoft Excel where providing such custom format strings is beyond their expertise. In our extensive case study of help forums of many programming languages and Excel, we found that both programmers and end-users struggle with these number transformations, but are able to easily express their intent using input-output examples. In this paper, we present a framework that can learn such number transformations from very few input-output examples. We first describe an expressive number transformation language that can model these transformations, and then present an inductive synthesis algorithm that can learn all expressions in this language that are consistent with a given set of examples. We also present a ranking scheme of these expressions that enables efficient learning of the desired transformation from very few examples. By combining our inductive synthesis algorithm for number transformations with an inductive synthesis algorithm for syntactic string transformations, we are able to obtain an inductive synthesis algorithm for manipulating data types that have numbers as a constituent sub-type such as date, unit, and time. We have implemented our algorithms as an Excel add-in and have evaluated it successfully over several benchmarks obtained from the help forums and the Excel product team
- …